27,186 research outputs found
Semismooth Newton Coordinate Descent Algorithm for Elastic-Net Penalized Huber Loss Regression and Quantile Regression
We propose an algorithm, semismooth Newton coordinate descent (SNCD), for the
elastic-net penalized Huber loss regression and quantile regression in high
dimensional settings. Unlike existing coordinate descent type algorithms, the
SNCD updates each regression coefficient and its corresponding subgradient
simultaneously in each iteration. It combines the strengths of the coordinate
descent and the semismooth Newton algorithm, and effectively solves the
computational challenges posed by dimensionality and nonsmoothness. We
establish the convergence properties of the algorithm. In addition, we present
an adaptive version of the "strong rule" for screening predictors to gain extra
efficiency. Through numerical experiments, we demonstrate that the proposed
algorithm is very efficient and scalable to ultra-high dimensions. We
illustrate the application via a real data example
A concave pairwise fusion approach to subgroup analysis
An important step in developing individualized treatment strategies is to
correctly identify subgroups of a heterogeneous population, so that specific
treatment can be given to each subgroup. In this paper, we consider the
situation with samples drawn from a population consisting of subgroups with
different means, along with certain covariates. We propose a penalized approach
for subgroup analysis based on a regression model, in which heterogeneity is
driven by unobserved latent factors and thus can be represented by using
subject-specific intercepts. We apply concave penalty functions to pairwise
differences of the intercepts. This procedure automatically divides the
observations into subgroups. We develop an alternating direction method of
multipliers algorithm with concave penalties to implement the proposed approach
and demonstrate its convergence. We also establish the theoretical properties
of our proposed estimator and determine the order requirement of the minimal
difference of signals between groups in order to recover them. These results
provide a sound basis for making statistical inference in subgroup analysis.
Our proposed method is further illustrated by simulation studies and analysis
of the Cleveland heart disease dataset
Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
A number of variable selection methods have been proposed involving nonconvex
penalty functions. These methods, which include the smoothly clipped absolute
deviation (SCAD) penalty and the minimax concave penalty (MCP), have been
demonstrated to have attractive theoretical properties, but model fitting is
not a straightforward task, and the resulting solutions may be unstable. Here,
we demonstrate the potential of coordinate descent algorithms for fitting these
models, establishing theoretical convergence properties and demonstrating that
they are significantly faster than competing approaches. In addition, we
demonstrate the utility of convexity diagnostics to determine regions of the
parameter space in which the objective function is locally convex, even though
the penalty is not. Our simulation study and data examples indicate that
nonconvex penalties like MCP and SCAD are worthwhile alternatives to the lasso
in many applications. In particular, our numerical results suggest that MCP is
the preferred approach among the three methods.Comment: Published in at http://dx.doi.org/10.1214/10-AOAS388 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
SCAD-penalized regression in high-dimensional partially linear models
We consider the problem of simultaneous variable selection and estimation in
partially linear models with a divergent number of covariates in the linear
part, under the assumption that the vector of regression coefficients is
sparse. We apply the SCAD penalty to achieve sparsity in the linear part and
use polynomial splines to estimate the nonparametric component. Under
reasonable conditions, it is shown that consistency in terms of variable
selection and estimation can be achieved simultaneously for the linear and
nonparametric components. Furthermore, the SCAD-penalized estimators of the
nonzero coefficients are shown to have the asymptotic oracle property, in the
sense that it is asymptotically normal with the same means and covariances that
they would have if the zero coefficients were known in advance. The finite
sample behavior of the SCAD-penalized estimators is evaluated with simulation
and illustrated with a data set.Comment: Published in at http://dx.doi.org/10.1214/07-AOS580 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Asymptotic oracle properties of SCAD-penalized least squares estimators
We study the asymptotic properties of the SCAD-penalized least squares
estimator in sparse, high-dimensional, linear regression models when the number
of covariates may increase with the sample size. We are particularly interested
in the use of this estimator for simultaneous variable selection and
estimation. We show that under appropriate conditions, the SCAD-penalized least
squares estimator is consistent for variable selection and that the estimators
of nonzero coefficients have the same asymptotic distribution as they would
have if the zero coefficients were known in advance. Simulation studies
indicate that this estimator performs well in terms of variable selection and
estimation.Comment: Published at http://dx.doi.org/10.1214/074921707000000337 in the IMS
Lecture Notes Monograph Series
(http://www.imstat.org/publications/lecnotes.htm) by the Institute of
Mathematical Statistics (http://www.imstat.org
Some results on optimal stopping problems for one-dimensional regular diffusions
For a type of employee stock option (ESO) and an American put option with a
barrier, we obtain closed-form formulae for the value functions and provide a
complete characterization for optimal stopping/continuation regions. Some
comparison principles for the critical levels and the value functions are
given. This work is inspired by the characterization of the value functions for
general one-dimensional regular diffusion processes developed in \cite{DK03} by
Dayanik and Karatzas.Comment: 35 page
Combining Multiple Clusterings via Crowd Agreement Estimation and Multi-Granularity Link Analysis
The clustering ensemble technique aims to combine multiple clusterings into a
probably better and more robust clustering and has been receiving an increasing
attention in recent years. There are mainly two aspects of limitations in the
existing clustering ensemble approaches. Firstly, many approaches lack the
ability to weight the base clusterings without access to the original data and
can be affected significantly by the low-quality, or even ill clusterings.
Secondly, they generally focus on the instance level or cluster level in the
ensemble system and fail to integrate multi-granularity cues into a unified
model. To address these two limitations, this paper proposes to solve the
clustering ensemble problem via crowd agreement estimation and
multi-granularity link analysis. We present the normalized crowd agreement
index (NCAI) to evaluate the quality of base clusterings in an unsupervised
manner and thus weight the base clusterings in accordance with their clustering
validity. To explore the relationship between clusters, the source aware
connected triple (SACT) similarity is introduced with regard to their common
neighbors and the source reliability. Based on NCAI and multi-granularity
information collected among base clusterings, clusters, and data instances, we
further propose two novel consensus functions, termed weighted evidence
accumulation clustering (WEAC) and graph partitioning with multi-granularity
link analysis (GP-MGLA) respectively. The experiments are conducted on eight
real-world datasets. The experimental results demonstrate the effectiveness and
robustness of the proposed methods.Comment: The MATLAB source code of this work is available at:
https://www.researchgate.net/publication/28197031
- …